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Abstract 


Advancements  in  human  brain  research  and  neuroscience  offer  great  promise  for  future 
joint  warfighting  capabilities.  As  the  national  security  environment  grows  in  complexity, 
weapon  system  designs  must  advance  to  provide  United  States  joint  warfighters  with  capabilities 
to  counter  increasingly  challenging  adversarial  threats.  It  is  hypothesized  that  by  the  year  2030, 
human  system  operators  will  be  “ cognitively  challenged ’  to  keep  pace  with  the  demands  of  the 
most  advanced  weapon  system  designs.  Some  are  calling  for  adoption  of  human  cognitive 
enhancement  coupled  with  flexible  autonomy— forming  human-machine  teams— as  a  means  to 
address  challenges  to  United  States  technical  superiority.  While  human  cognitive  enhancement 
may  hold  promise,  the  practice  also  raises  questions  and  concerns.  The  potential  for  ethical 
implications  is  a  matter  United  States  policymakers  must  afford  consideration.  Some 
theoreticians  go  so  far  as  to  envision  dynamic  “ cognitive  coupling ”  of  human  system  operators 
with  flexible  autonomous  machines  via  brain  computer  interfaces.  The  resultant  human-machine 
team  is  analogous  with  the  Air  Force  vision  for  “Airman-machine  teaming.”  Airman-machine 
teaming  raises  questions  regarding  the  imperative  for  maintaining  human  control  over 
autonomous  weapon  systems.  Control  of  lethal  force  demands  special  attention.  Concerns  are 
further  heightened  with  the  possibility  of  injecting  human  cognitive  enhancement  into  the 
environment.  This  research  paper  explores  ethical  implications  associated  with  adoption  of 
human  cognitive  enhancement.  As  the  Air  Force  considers  adopting  neurotechnologies  and 
pursuing  Airman-machine  teaming,  this  paper  argues  continuous  study  and  policy  debate  of  the 
potential  ethical  implications  are  warranted. 
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Introduction 


As  envisioned  by  the  Joint  Staff,  and  as  expressed  in  the  Joint  Operating  Environment 
2035  (JOE  2035),  the  United  States  (US)  will  face  a  most  complex  security  atmosphere  in  the 
year  2035. 1  The  future  security  environment  will  increasingly  be  characterized  by  disorder  and 
chaos.  Some,  such  as  Antoine  Bosquet,  argue  advances  in  science  and  technology  promise  to 
bring  order  to  this  “chaoplexic”  warfare  environment.2  Irrespective  of  the  potential  for  scientific 
and  technological  advances,  Airmen  are  destined  to  face  greater  personal  challenges  to  their 
individual  human  abilities:  human  cognitive  performance  is  no  exception. 

The  US  Department  of  Defense  (DoD)  anticipates  the  joint  warfighter,  the  human  system 
operator,  will  be  confronted  with  challenges  to  cognitive  function  beyond  “normal”  human 
capabilities.  As  envisioned,  these  future  challenges  to  human  cognitive  capabilities  will  eclipse 
that  which  is  generally  considered  “natural”  human  capacity.  While  future  wars  will  continue 
to  feature  battles  of  physical  feats,  the  multi-domain  battlespace  of  the  future  will  expand  to 
include  competition  within  a  new  “mentalsphere”  dimension— a  cognitive-machine  space- 
seeking  to  establish  cognitive  supremacy  of  shared  human-machine  capabilities. 

As  Carl  von  Clausewitz  notes  in  On  War,  “war  is  an  act  of  force  to  compel  our  enemy  to 
do  our  will.”3  Clausewitz’ s  dictum  highlights  the  application  of  physical  force  in  war  as  a  means 
to  achieve  the  ultimate  ends:  shifting  of  an  adversary’s  mental  attitude  and  psychological 
disposition.  In  Clausewitz’ s  view,  the  use  of  physical  force  in  war  is  to  compel  one’s  enemy- 
bending  will  and  shifting  mental  intent.  Nowhere  is  this  mindset  more  prominent  than  within  the 
sphere  of  cognitive  thoughts  and  mental  consciousness.  Clausewitz’s  thinking  reflects  an 
understanding  of  the  importance  of  human  cognitive  capabilities  in  war.4 


As  warfare  evolves  unpredictably,  with  chaos  and  complexity  increasingly  defining  the 
future  multi-domain  battlespace,  discoveries  associated  with  neuroscience  and  advancements  in 
neurotechnology  offer  promise  for  maintaining  US  warfighting  competitive  advantage. 

Neuroscience  is  focused  on  unlocking  the  mysteries  of  the  human  brain.  According  to 
the  Society  for  Neuroscience,  neuroscientists  are  focused  on  gaining  a  better  understanding  of 
the  complexities  associated  with  the  human  brain  and  for  uncovering  the  secrets  to  unleash  the 
full  potential  of  humankind.5  It  is  the  view  of  neuroscientists  that  the  human  brain  has  more 
computing  capacity  than  a  modern  supercomputer-capabilities  valued  by  US  warfighters. 

As  complicated  as  these  challenges  are,  neuroscience  is  not  confined  simply  to 
understanding  the  physical  aspects  of  the  human  brain.  Perhaps  more  daunting  is  the  task  of 
deciphering  the  billions  of  nerve  cells,  tissue  fibers,  chemical  compositions  and  electrical 
impulses  making  up  the  human  nervous  system.  Neuroscientists  endeavor  to  understand  the 
mental  complexion  of  the  human  mind:  our  ability  to  “shape  thoughts,  beliefs,  hopes,  dreams, 
and  imaginations.”6  From  a  cognitive  enhancement  perspective,  neuroscience  is  concerned  as 
much  with  understanding  the  capabilities  and  functions  of  the  brain,  and  if  not  more  so,  gaining 
an  understanding  of  the  inner  workings  of  the  mind. 

Joined  with  neuroscience  is  neurotechnology— the  implements  of  brain  science.  Recent 
advancements  in  neurotechnology  hold  promise  for  putting  neuroscience  into  practice.  Present- 
day  advancements  in  neurotechnology  herald  to  the  mid-1960s  when  Grey  Walter  first 
demonstrated  the  application  of  a  non-invasive  encephalogram  (EEG).7  It  was  not  long 
thereafter,  in  the  early  1970s,  that  the  Defense  Advanced  Research  Agency  (DARPA)  got 
involved  and  first  funded  development  of  neurotechnology  in  the  form  of  a  brain-computer 
interface  (BCI).8  It  is  noted,  DARPA’ s  interests  in  advancing  neurotechnology  in  support  of 
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warfighter  applications  has  grown  steadily  ever  sense.  DARPA  recently  issued  Duke 
University’s  Center  for  Neuroengineering  a  $26  million  dollar  contract  as  part  of  DARPA’ s 
Brain-Machine  Interfaces  Program  to  “develop  new  technologies  for  augmenting  human 
performance  by  accessing  the  brain  in  real  time  and  integrating  the  information  into  external 
devices.”9  The  stated  DARPA  goal  sounds  very  much  like  the  AF  vision  for  human  cognitive 
enhancement  and  Airman-machine  teaming.  This  level  of  support  from  DARPA  demonstrates 
interest  in  developing  brain-machine  interfaces  to  gain  advantage  of  directly  interfacing  human 
brains  with  machines,  devices  John  Edwards  refers  to  in  The  Geeks  of  War  as  “neurorobots.”10 

In  2015,  no  fewer  than  thirty  thousand  scientists  and  industry  professionals  gathered  in 
Chicago  for  the  annual  Society  of  Neuroscience  conference:  brain  function  and  cognitive 
enhancement  were  hot  topics  that  year. 1 1  Indicative  of  this  level  of  interest,  ten 
neurotechnologies  were  showcased  specifically  relating  to  human  cognitive  enhancement.  All 
ten  would  appear  to  have  warfighter  interests,  and  applicability: 12 

1 .  Big  data-enhanced  diagnostics  and  treatments 

2.  Brain-computer  interfaces  for  device  control 

3.  Real-time  neuromonitoring  (plus  robotic  aids) 

4.  Neurosensor-based  vehicle  operator  systems 

5.  Cognitive  training  videogames 

6.  Brain-responsive  computing  systems 

7.  Virtual  reality  treatments  (especially  in  conjunction  with  EEG  and/or  tDCS*) 13 

8.  “Mindful”  wearables 

9.  Collaborative  cognitive  simulations 

10.  Electrical  and  magnetic  brain  stimulation 
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Given  the  level  of  DARPA’s  investment  in  neuroscience,  one  can  argue  the  DoD  clearly 
sees  a  need  to  pursue  “leap  ahead”  neurotechnologies  to  maintain  US  technical  superiority.  As 
the  AF  moves  closer  towards  adoption  of  neuroscience  and  neurotechnology  to  meet  future 
complex  challenges  and  counter  growing  adversarial  threats,  an  assessment  of  potential  ethical 
implications  must  be  given  greater  consideration.  In  particular,  as  the  AF  looks  to  refine 
operational  concepts  under  the  auspices  of  Airman-machine  teaming,  notions  of  human  cognitive 
enhancement  in  particular  deserve  scrutiny  from  an  ethical  perspective.  With  an  awareness  for 
the  complexities  envisioned  in  the  JOE  2035  rapidly  approaching,  US  policymakers  could  soon 
be  confronted  with  calls  to  expedite  adoption  of  neuroscience  and  neurotechnologies  without 
having  given  potential  ethical  implications  due  consideration.  With  on-going  research  advancing 
at  a  rapid  pace,  now  is  the  time  to  begin  an  examination  of  the  potential  ethical  implications 
associated  with  adopting  human  cognitive  enhancement. 

The  process  of  policy  formulation  need  not  await  in  anticipation  of  research  findings  and 
scientific  facts  to  initiate  the  discussion  and  debate.  As  James  Browning  highlights  in,  “ Leading 
at  the  Strategic  Level  in  an  Uncertain  World,”  senior  leaders  often  will  not  have  all  the  answers 
before  they  must  start  working  to  identify  solutions  for  “wicked”  problems. 14  Gaining  a  better 
appreciation  of  volatile,  uncertain,  complex  and  ambiguous  (VUCA)  problems  is  valuable  in  and 
of  itself.  Answers  in  these  situations  do  not  come  easily,  nor  quickly. 

The  time  has  arrived  for  US  policymakers  to  begin  the  discussion  and  initiate  debate 
concerning  potential  ethical  implications  associated  with  human  cognitive  enhancement,  and  in 
the  larger  context,  human-machine  teams  coupled  with  flexible  autonomy.15 

As  envisioned,  flexible  autonomy  represents  a  symbiotic  relationship  between  human  and 
machine  (Airman- machine  teaming  is  just  one  manifestation  of  this  vision  from  an  AF 
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perspective).  Scientific  discovery  and  associative  policy  development  share  a  interdependent 
and  reciprocal  relationship.  Scientific  discovery  must  inform  policy  development;  policy 
development  must  guide  scientific  discovery.  This  research  paper  highlights  the  need  for  an  in- 
depth  study  examining  potential  ethical  implications  concerning  human  cognitive  enhancement 
in  association  with  Airman-machine  teaming. 

As  the  DoD  and  Joint  Staff  devise  contingency  plans  to  address  potential  future  military 
conflicts,  US  policymakers  must  assess  research  discoveries  and  scientific  possibilities  to  meet 
the  future  security  needs  of  the  joint  warfighting  force.  Recent  research  discoveries  concerning 
the  human  brain  are  pushing  scientific  possibilities  for  adopting  human  cognitive  enhancement. 
What  some  today  assert  are  future  potentialities,  tomorrow  they  may  insist  the  same  are  absolute 
necessities.  If  policy  debate  is  not  initiated,  adoption  of  human  cognitive  enhancement  could 
become  “de  facto  ...fait  accompli!" 

Thesis 

Airman-machine  teaming  is  one  personification  of  flexible  autonomy.  Through  human 
cognitive  enhancement,  neuroscientists  endeavor  to  augment  “normal”  human  capabilities 
through  neurotechnology  to  bring  about  improved  human  performance.  This  research  paper 
argues  potential  ethical  implications  associated  with  human  cognitive  enhancement  in 
association  with  flexible  autonomy  (Airman-machine  teaming)  are  not  being  adequately 
addressed  by  US  policymakers.  As  such,  the  AF  is  at  risk  for  allowing  science  to  proceed  . . .  and 
precede  . . .  policy  regarding  potential  ethical  implication  of  human  cognitive  enhancement. 

Neuroscientific  research  on  the  human  brain  is  proceeding  at  a  rapid  pace.  Arguably, 
neuroscientific  discovery  and  neurotechnological  advancements  are  proceeding  ahead  of 
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discourse,  debate  and  policy.  While  uncertainty  must  not  hold  science  hostage,  scientists  share 
in  responsibility  for  enlightening  the  discourse  and  fostering  public  policy  debate. 

The  AF  has  articulated  science  and  technology  visions  and  future  operating  concepts,  yet 
policy  concerning  potential  ethical  implications  associated  with  flexible  autonomy  and  human 
cognitive  enhancement  are  lacking— ethics,  principles  and  core  values  hang  in  the  balance. 

In  furtherance  of  Airman-machine  teaming,  this  paper  calls  for  the  AF  to  authorize  a 
study  effort  with  the  goal  of  establishing  official  policy  and  setting  operational  guidelines 
addressing  potential  ethical  implication  associated  with  human  cognitive  enhancement  and 
flexible  autonomy.  The  study  effort  should  seek  to  provide  definitive  answers  to  the  following: 

1)  Are  there  ethical  implications  associated  with  human  cognitive  enhancement; 

2)  Does  human  cognitive  enhance  jeopardize  the  concept  of  human  control? 

In  support  of  these  arguments,  this  research  paper  examines  DoD  and  AF  policy  and 
pronouncements  regarding  autonomy,  and  science  and  technology  in  general.  In  particular,  areas 
concerning  ethics  relating  to  human  cognitive  enhancement  were  a  focus  of  research.  While 
considerable  investment  is  being  made  in  neuroscience  and  neurotechnology,  as  noted  by  the 
DARPA  example  cited,  there  is  little  evidence  ethics  is  receiving  seriously  considerations.  As 
this  research  paper  notes,  neither  the  2012  nor  the  2016  Defense  Science  Board  reports  on 
autonomy  address  ethics.  Further,  discussion  is  presented  regarding  questions  and  concerns 
associated  with  jeopardizing  independent  human  control,  accountability  and  responsibility  as  a 
result  of  implementation  of  flexible  autonomy  in  association  with  human  cognitive  enhancement. 
A  discussion  of  possible  consequences  to  the  Law  or  Armed  Conflict  and  Just  War  theory  is 
presented  for  contemplation.  In  summary,  this  research  shows  policy  debate  regarding  potential 
ethical  implications  concerning  flexible  autonomy  and  human  cognitive  enhancement  is  lacking. 
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A  Glimpse  Into  The  Future 


“ Neuroweapons ?” 

The  US  Air  Force  Future  Operating  Concept  (AFFOC)  establishes  a  vision  for  how 
Airmen  and  Airpower  will  meet  the  security  challenges  depicted  in  the  JOE  2035. 16  Scenes  in 
the  AFFOC  depict  future  weapon  systems  (machines)  and  human  system  operators  (Airmen) 
teamed  as  interdependent  partners  via  intelligent  technologies  fighting  across  multi-domain 
environments.  As  an  inspiring  and  aspirational  document,  the  AFFOC  is  afforded  literary 
license  for  omitting  some  details  for  how  human  system  operators  will  achieve  the  seemingly 
science  fiction  feats  illustrated  in  the  vignettes.17  Not  to  dismiss  these  futuristic  ideas  and 
concepts,  but  as  the  AF  plans  for  organizing,  training  and  equipping  the  “ The  World’s  Greatest 
Air  Force,”  reality  eventually  enters  the  picture  through  science  and  technology  investments  and 
funded  acquisition  programs.18  19  With  interest  in  human  cognitive  enhancement  and  Airman- 
machine  teaming  heightened,  now  is  an  opportune  time  to  engage  in  theoretical  inquiry  regarding 
the  potential  of  ethical  implications.  Philosophical  questions  are  best  considered  in  advance  of 
programmatic  decisions  allowing  sufficient  time  for  well-reasoned  discussion  and  balanced 
debate  before  reality  presents  the  problem  in  demanding  terms  hastening  well-reasoned 
considerations. 

It  is  hypothesized  that  by  the  year  2030,  human  cognitive  capabilities  will  be  challenged 
to  keep  pace  with  the  “enormous  data  volumes,  processing  capacities,  and  decision  speeds” 
associated  with  future  weapon  systems.20  We  already  see  evidence  of  this  future  reality.  Past 
studies  exploring  mental  workload  and  task  demands  levied  on  human  system  operators  of 
command  and  control  (C2)  systems  support  this  hypothesis.21  The  recent  2008  National 
Academy  of  Sciences  publication  on  emerging  technologies  highlighted  possible  military 
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applications  of  neuroscience  for  the  purposes  of  enhancing  human  cognitive  functionality.22 
Specifically  mentioned  in  the  2008  report  are  recommendations  for  the  US  Army  to  increase 
funding  for  scientific  research  associated  with  transcranial  magnetic  stimulation  (TMS)  and 
examine  modem  pharmaceutical  technologies  (e.g.  modafinil,  methylphenidate,  etc.).  It  is  the 
view  of  the  National  Academy  of  Sciences  these  modern  neurotechnologies  may  hold  promise 
with  military  utility  (e.g.  suppressing  effects  of  sleep  deprivation  and  increasing  alertness).23  In 
a  separate,  but  somewhat  related  work,  Rachel  Wurzman  and  James  Giordano  note  the  potential 
for  development  of  defense-oriented  neurotechnologies  to  create  “neuroweapons”24 

With  this  view  of  the  future  in  mind,  it  is  easy  for  one  to  visualize  seeds  of  requirements 
germinating  as  expressions  of  future  operational  imperatives— ruminations  taking  shape  in  the 
form  of  formal  joint  warfighting  operational  requirements.  The  vision  for  Airman-machine 
teaming  and  flexible  autonomy  resonates  with  this  view. 

The  US  is  pursuing  advancements  in  unmanned  systems,  robotics,  and  autonomy  as  ways 
and  means  to  meet  future  security  needs.  The  AF  science  and  technology  strategy  does  not 
envision  supplanting  humans  completely,  yet  the  AF  strategy  envisions  capitalizing  on  and 
leveraging  the  unique  strengths  and  capabilities  of  human  system  operators  seamlessly  matched 
with  the  distinctive  benefits  of  flexible  autonomous  machines.25 

A  Human  Touch 

Maintaining  Balance  . . .  Perspective 

The  uniquely  human  capacity  for  independent  thought,  judgement  and  reasoning  . . .  must 
be  retained.  However,  at  the  same  time,  it  is  this  very  human  capability  for  cognitive  function 
that  is  potentially  the  limiting  factor  in  a  future  complex  world.  The  challenge  as  seen  is  how 
best  to  effect  Airman-machine  teaming  without  violating  human  autonomy  or  undermining 
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human  control.  Nowhere  is  this  concern  more  important  than  in  policy  discussions  surrounding 
the  use  of  lethal  force. 

When  it  comes  to  Airman-machine  teaming  and  implementation  of  autonomy,  DoD 
policy  regarding  the  use  of  lethal  force  is  clearly  stated.  Deputy  Secretary  of  Defense  Robert 
Work  states  the  US  position  unequivocally,  “we  believe  strongly  that  humans  should  be  the  only 
ones  to  decide  when  to  use  lethal  force.”  Secretary  Work’s  comments  are  not  referring  to  human 
physical  control,  but  more  importantly  that  of  human  mental  attitude  and  judgement— the 
uniquely  human  ability  to  reason  and  decide.  In  his  statement,  Secretary  Work  foresaw  a  time  in 
the  not  too  distant  future  when  the  US  will  be  confronted  with  the  question  of  human  cognitive 
enhancement,  “Now  our  adversaries  quite  frankly  are  pursuing  enhanced  human  operations  and 
it  scares  the  crap  out  of  us,  really.  We’re  going  to  have  to  have  a  big,  big  decision  on  whether 
we’re  comfortable  going  that  way.”26  For  the  foreseeable  future,  US  policy  will  most  likely 
continue  to  require  direct  human  operator  involvement  in  the  use  of  lethal  force,  however,  a  time 
is  nearing  when  the  question  of  human  cognitive  enhancement  must  be  addressed.  As  the  DoD 
assesses  the  ramifications  of  human-machine  teaming  and  interactive  cognitive  partnering  is 
considered  along  with  human  cognitive  enhancement,  questions  and  understanding  concerning 
“control”  of  lethal  force  will  grow  in  importance.  These  questions  will  demand  as  much  a 
scientific  and  technological  debate  as  they  will  a  philosophical  discussion  concerning  moral 
imperatives  and  ethical  implications.  The  ethical  implications  of  such  questions  have  far 
reaching  implications  relating  to  US  policy,  International  law,  and  theory  guiding  Just  War. 

As  the  DoD  plans,  programs,  and  prepares  the  military  force  of  the  future  ...  to  meet  the 
security  challenges  of  the  future  . . .  the  US  must  not  wait  to  address  questions  concerning 
potential  ethical  implications  surrounding  human  cognitive  enhancement. 
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A  New  War  Order 


The  “Big,  Big”  Question  Before  Us 

In  November  2014,  US  Secretary  of  Defense  Chuck  Hagel  announced  creation  of  the 
Defense  Innovation  Initiative  (“Third  Offset”)  strategy  to  advance  technologies  including 
robotics,  artificial  intelligence,  autonomous  systems,  and  big  data.27  Answering  this  call  for 
advancing  technologies  to  meet  future  demands,  the  US  Air  Force  year  2030  strategy,  America ’s 
Air  Force:  A  Call  to  the  Future ,  recognizes  unmanned  and  autonomous  weapon  systems  as 
potential  game-changing  technologies.28  The  Air  Force  envisions  the  application  of  flexible 
autonomy  to  “form  a  central  tenet  for  future  human-autonomy  interaction.”29  Flexible  autonomy 
will  come  to  represent  the  state  of  interaction  between  unmanned  systems  and  human  system 
operators:  the  two  working  together— interdependently— in  a  symbiotic  relationship.30 

Providing  some  definition  and  context  to  the  concept  of  human-machine  teaming,  Deputy 
Defense  Secretary  Work  adds,  “Human-machine  collaboration  is  allowing  a  machine  to  help 
humans  make  better  decisions  faster.  That  is  a  big,  big  difference.  There  is  an  artificial 
intelligence  (AI)  bias  right  now  generally  in  the  community  [i.e.  using  unmanned  systems  to 
replace  humans,  rather  than  augment  them].  But. .  .automated  systems  use  algorithms  based  on 
old  data,  [and]  we’re  up  against  a  thinking  adversary  that  is  changing  strategies  all  the  time.”  In 
Secretary  Work’s  remarks,  one  finds  concern  for  the  adversarial  threat  developing  as  a  prime 
motivating  factor  for  the  US  to  advance  concepts  of  human-machine  teaming,  and  the 
capabilities  envisioned  through  such  partnership. 

“Thinking  machines’,  robotics  embedded  with  artificial  intelligence  and  augmenting,  off¬ 
setting  human  ‘handling  cognitive  task,”  is  how  Erik  Brynjolfsson  and  Andrew  McAfee  describe 
the  future  in  The  Second  Machine  Age.31  The  idea  is  not  about  machines  replacing  humans;  it  is 
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about  machines  and  humans  working  in  tandem,  cognitively  coupled  as  partners.  The  essence  of 
this  vision  is  analogous  with  the  AF  concept  of  Airman-machine  teaming.  As  military  futurist 
Paul  Scharre  imagines  the  potential  for  human-machine  teaming,  “It’s  actually  not  an  either-or. 
Like  the  mythical  centaur,  we  can  harness  inhuman  speed  and  power  to  human  judgement.  We 
can  combine  machine  precision  and  reliability,  human  robustness  and  flexibility.”32  As  Scharre 
describes  the  potential  of  human-machine  teaming,  “. . .  it’s  not  just  about  manned  and  unmanned 
vehicles  moving  together,  it’s  also  about  computers  and  humans  helping  each  other  to  think. 
That’s  “cognitive  teaming.”33 

Given  descriptions  provided  by  Work  and  Scharre,  one  can  begin  to  see  how  advances  in 
human-machine  teaming  can  make  it  difficult  to  separate  the  thoughts  of  the  human  from  actions 
of  the  machine.  Will  there  be  any  separation  between  human  cognitive  thought  and  machine 
physical  action.  Likewise,  one  might  ask  how  this  close  association,  an  interdependent, 
symbiotic  relationship  between  living  human  beings  and  autonomous  machines  may  affect 
human  thought  and  decision  making. 

Is  Cognitive  Enhancement ...  Already  A  Foregone  Conclusion? 

Establishing  conditions  necessary  for  adoption  of  unmanned  and  autonomous  systems,  in 
support  of  Airman-machine  teaming,  the  Air  Force  Technology  Horizons  matter-of-factly  states, 
“Humans  and  machines  will  need  to  become  far  more  closely  coupled  through  improved  human- 
machine  interfaces  and  by  direct  augmentation  of  human  performance .”34  The  Air  Force  Future 
Operating  Concept  (AFFOC)  calls  for  the  development  of  human-machine  systems  in  support  of 
the  year  2035  force  of  the  future  operations.35  36  As  such,  the  AFFOC  notes,  “As  a  force 
development  concept,  the  AF  Future  Operating  Concept  is  subject  to  testing,  experimentation, 
evaluation  and  assessment  to  validate  its  ideas  and/or  suggest  better  alternatives.”37  Cautioned  in 
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the  AF  strategy,  America ’s  Air  Force:  A  Call  to  the  Future,  is  a  warning  “not  to  allow 
technology  to  outpace  legal,  moral,  and  doctrinal  considerations.”38  In  light  of  Secretary  Works’ 
comments  noted  earlier,  and  as  expressed  in  the  Air  Force  Technology  Horizons,  it  is  prudent  for 
US  policymakers  and  the  DoD  to  heed  a  cautionary  note  regarding  ethical  implications 
associated  with  human  cognitive  enhancement. 

Technology  Horizons  acknowledges  concerns  associated  with  some  forms  of 
neuroscientific  techniques  and  human  cognitive  enhancement,  “while  such  methods  may  appear 
inherently  distasteful,  potential  adversaries  may  be  entirely  willing  to  make  use  of  them.”39 
Acknowledging  concerns  exist,  it  is  important  to  evaluate  and  assess  the  ethical  implications  and 
policy  considerations  associated  with  adoption  of  neurotechnology  relating  to  human  cognitive 
enhancement  and  not  allow  a  drive  to  match  adversaries  push  the  US  to  violate  ethical  principles. 

The  Head  of  the  Matter 

Intelligent  machines  seamlessly  integrated  with  humans  -  maximizing  mission 
performance  in  complex  and  contested  environments40 

Greg  L.  Zacharias,  PhD,  SES,  Chief  Scientist  of  the  Air  Force 

Controlling  Thought:  Human  or  Machine 

As  Department  of  Defense  Directive  3000.09  states,  “Autonomy  in  Weapon  Systems,”  it 
is  DoD  policy  for  autonomous  (semi-autonomous  or  flexible  autonomous)  weapon  systems  to 
"...  be  designed  to  allow  commanders  and  operators  to  exercise  appropriate  levels  of  human 
judgement  over  the  use  of  force.”41  As  the  title  of  DoDD  3000.09  implicates,  the  primary 
subject  of  the  directive  surrounds  the  use  of  autonomy  in  weapon  systems.  Thus,  the  directive  is 
principally  concerned  with  autonomy,  the  machine  aspect  of  the  weapon  system,  and  not  the 
human  operator  element  of  control.  While  the  directive  stipulates  human  operator  involvement, 
or  “control  by,”  the  directive  is  silent  regarding  the  cognitive  component  of  human  judgement. 
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From  a  cognitive  perspective,  it  would  appear  there  is  ambiguity  surrounding  what 
constitutes  human  control.  This  uncertainty  adds  to  the  unease  regarding  adoption  of  autonomy 
with  Airman-machine  teaming,  coupled  with  the  potential  for  human  cognitive  enhancement.  In 
light  of  the  previous  discussion  regarding  the  real  and  foreseeable  potential  for  human  cognitive 
enhancement,  the  AF  must  assess  if  there  are  resultant  ethical  implications  by  effecting  change  to 
a  human  system  operator’s  cognitive  state.  Does  human  cognitive  enhancement  sufficiently  alter 
or  shift  mental  state  thus  jeopardizing  human  control? 

Calls  for  adoption  of  neurotechnology  to  advance  human  cognitive  capabilities  demand 
prudent  consideration.  Recognizing  human  judgement  is  correlated  with  reasoning  and  decision 
making,  and  is  a  joined  facet  of  cognition,  arguably,  human  cognitive  enhancement  could  have 
some  impact  on  an  individual’s  ability  to  reason  clearly.  The  concern  is:  can  human  cognitive 
enhancement  impair  judgement  sufficiently  and  negatively  to  violate  the  concept  of  self  and 
human  control.  As  much  remains  unknown  regarding  the  deeper  functions  of  the  human  brain, 
scientists  must  resist  the  allure  of  “technological  imperative”  to  use  technology  simply  because  it 
is  available.42  Even  allowing  for  greater  understanding  of  the  structures  of  the  brain,  as  a  result 
of  advancements  in  brain  imaging  and  neuro  sensory  technologies,  there  is  still  much  speculation 
regarding  mental  cognition  and  precise  brain  function.  This  is  especially  true  concerning  the 
phenomenon  of  thought  and  consciousness.43 

In  their  examination  of  autonomous  weapons  systems,  Michael  Horowitz  and  Paul 
Scharre  produced  a  report,  Meaningful  Human  Control  in  Weapons  Systems:  A  Primer, 
addressing  the  question  for  what  substantiates  “meaningful”  human  control.  In  their  report, 
Horowitz  and  Scharre  propose  two  conditions  relating  to  the  human  operator: 

1.)  Humans  make  informed  and  conscious  decision. 
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2.)  Information  available  is  sufficient  to  ensure  action  lawful. 


Considering  the  question  of  impact  of  cognitive  enhancement  on  the  reasoning  ability  of 
human  subjects,  the  stipulation  for  informed  and  conscious  decision  deserves  greater 
examination. 

Much  mystery  remains  concerning  dualism  and  the  connections  between  brain  and  mind. 
As  David  Chalmers  equates  the  situation  in  the  Journal  of  Consciousness  Studies,  demonstrating 
causality  between  brain  function  and  conscious  thoughts  remains  one  of  the  “hard  problems”  of 
neuroscience.44  Recognizing  that  uncertainty  and  risk  are  ever-present  facets  of  scientific 
discovery,  scientist  must  approach  application  of  neurotechnology  with  trepidation. 

Over  the  course  of  the  past  few  decades  there  has  been  much  research  into  the  depths  of 
the  human  brain.  Medical  discoveries  have  revealed  many  mysteries  surrounding  the  features 
and  structures  of  the  human  brain.  As  an  anatomical  organ,  the  human  brain  is  fairly  well 
documented.  However,  when  it  comes  to  the  human  mind,  less  is  understood.  The  phenomenon 
of  consciousness  in  particular  remains  much  a  mystery.  As  James  Giordano  argues,  this  set  of 
circumstances  should  not  dissuade  neuroscience  research  and  development.45  Rather,  Giordano 
favors  pursuing  research  since  the  outcomes  provide  a  foundation  of  knowledge  from  which  to 
develop  understanding  for  how  to  handle  unexpected  outcomes.46  As  the  AF  considers 
opportunities  for  advancing  human  cognitive  enhancement,  in  association  with  Airman-machine 
teaming,  the  sentiments  of  Giordano  ring  true:  research  and  discovery  must  inform  policy 
formulation  . . .  policy  must  guide  science  and  technology  efforts. 

The  Evolution  of  “Autohumachine” 

The  dynamism  of  an  “adversarial-response  dilemma,”  the  feeling  of  an  imperative  and 
compunction  to  respond  to  threats  in  like  mind  can  be  strong  motivating  forces  pushing  US 
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decision  makers  to  pursue  new  technology  such  as  human  cognitive  enhancement.  Robotics, 
autonomy,  artificial  intelligence,  and  unmanned  weapon  systems  are  just  some  of  the  advanced 
technologies  often  highlighted  as  key  to  the  US  maintaining  technological  superiority  over  our 
adversaries.47  While  the  names  of  these  futuristic  technologies  can  convey  a  perception  of 
human-less  or  operator-less  weapons  systems,  as  the  DoD  Unmanned  Systems  Integrated 
Roadmap  attests,  nothing  could  be  farther  from  reality.48  Even  the  most  advanced  application  of 
autonomy  envisions  some  form  of  interaction  between  human  and  machine. 

As  Dr.  Vladimir  Djapic  highlights  in  the  monograph,  Optimizing  the  Human  Element  in 
the  Age  of  Autonomy,  achieving  optimization  of  autonomous  weapon  systems  is  predicated  on 
maximizing  the  inherent  strengths  of  human  operators  and  exploiting  advanced  technical 
capabilities  of  autonomous  machines.49  The  2012  DoD  Defense  Science  Board  task  force  report 
on  The  Role  of  Autonomy  in  DoD  Systems  supports  this  assessment.50  Interestingly,  neither  the 
2012  DoD  Defense  Science  Board  report  on  autonomy,  nor  the  more  recent  2016  DSB  report, 
addresses  the  question  of  human  cognitive  enhancement.51  Further,  these  reports  are  silent  on 
the  subject  of  ethical  considerations  regarding  human-machine  teaming.  A  lack  of  discussion  by 
the  DSB  concerning  ethics  is  further  evidence  demonstrating  the  need  for  study  and  debate. 

Moving  Beyond  Research  to  Policy  Considerations 

Dynamic  Human-Machines  ...  Brain-Based  Designs 

As  Steve  Murray  and  Matthew  Yanagi  posit  in  Transitioning  Brain  Research:  From 
Bench  to  Battlefield,  future  autonomous  system  designs  will  likely  be  based  on  elements  of 
human  neuroscience  research  including  human-machine  interactions.  Future  human-autonomous 
systems  will  likely  include  real-time  human  operator  physiological  state  sensing,  dynamic  task 
allocation,  and  bi-directional  feedback.52  Murray  and  Yanahi  highlight  some  of  the  most 


15 


promising  neuroscience-related  capabilities  associated  with  man-machine  teaming  having 
potential  for  national  security  application: 

•  Human  cognitive  performance:  gaining  better  understanding  of  processes 
involved  with  memory,  emotion,  and  reasoning  which  can  lead  to  improved  task 
design  and  decision  support  systems. 

•  Training  efficiency:  enabling  rapid  mastery  of  knowledge  with  improved 
retention. 

•  System  engineering:  include  technologies  supporting  shared-initiative  problem 
solving  between  humans  and  machines. 

In  order  for  scientists  to  advance  neuroscience  research  discoveries  to  support  human- 
machine  teaming,  engineers  must  explore  creating  new  architectures  supporting  cognitive  task 
allocation  and  dynamic  balancing  between  human  system  operators  and  intelligent  autonomous 
machines.  As  Murray  and  Yanagi  highlight  in  their  writing,  while  scientific  discoveries  and 
technological  advances  promise  potential,  they  also  present  a  hosts  of  unique  challenges  that 
must  be  addressed  in  order  to  successfully  transition  neuroscience  to  future  weapon  system 
designs.53  In  order  to  develop  systems  supporting  shared  information  processing  and  dynamic 
association  between  human  (brain-based)  and  machine  (flexible  autonomy),  the  following 
transition  challenges  must  be  addressed: 

•  Minimize  disruption:  scientific  and  research  community  should  adopt  a  gradual 
“stepwise”  approach  to  introduce  and  adopt  brain-based  technologies. 

•  Define  agency  and  responsibility:  initiate  multi-disciplinary  dialogue  to  establish 
and  define  rules  and  norms  guiding  implications  of  brain-based  system  designs. 
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•  Design  for  transparency:  as  brain-based  designs  will  reflect  operator  cognitive 
state,  standards  must  be  developed  to  protect  privacy  and  provide  system  surety. 

As  highlighted  in  the  National  Research  Council  report,  Emerging  Cognitive 
Neuroscience  and  Related  Technologies,  potential  advancements  in  human-machine  and  brain- 
based  systems  holds  promise  for  improving  future  mission  performance.54  However, 
implementation  of  brain-based  human-machine  designs  demands  consideration  early  in  the 
design  process  for  potential  ethical  implications.  Murray  and  Yanagi  suggests  “neuro  research” 
scientists  should  look  to  recent  examples  from  human  factors  engineering  (HFE)  and  human 
systems  integration  (HSI)  when  assessing  the  potential  for  integration  of  human-machine 
systems.  Two  pertinent  suggestions  the  AF  should  consider  in  relationship  to  Airman-machine 
teaming  includes: 

•  Engage  user  community  early  to  identify  applications  for  technologies 

•  Adopt  iterative  approach  to  incremental  testing  in  operational  environment 
Cognitive  Enhancement ...  Just  War  Impact 

As  the  DoD  and  AF  consider  human-machine  teaming,  US  policymakers  must  assess 
how  human  cognitive  enhancement  may  negatively  impact  Just  War  theory.  Principles 
governing  human  conduct  in  war  are  rooted  in  Just  War  theory  as  reflected  in  the  Law  of  Armed 
Conflict .55  Principles  of  Just  War  are  largely  based  on  two  elements:  Jus  ad  helium  (declaration 
of  war)  and  Jus  in  hello  (conduct  of  war).  Jus  in  hello  captures  the  essence  of  one’s  individual 
responsibility  in  war,  to  include  responsibility  for  exercising  sound  judgement  over  the 
application  of  lethal  force. 

With  respects  to  human  cognitive  enhancement,  it  is  jus  in  hello  which  is  pertinent  to  the 
discussion  regarding  potential  ethical  implications  surrounding  human  cognitive  enhancement. 
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As  human  cognitive  enhancement  has  some  potential  to  alter  one’s  perceptions  (of  reality), 
thereby  affecting  associative  judgements,  could  the  resultant  impairment  potentially  jeopardize 
human  autonomy  and  bring  into  question  assignment  of  responsibility? 

Individual  responsibility  according  to  jus  in  bello  is  governed  by  three  explicit 
conditions:  discrimination,  proportionality,  and  necessity.56  To  maintain  the  integrity  of  Jus  in 
bello,  and  individual  responsibility,  all  three  conditions  must  exist  without  impairment  or 
degradation: 

•  Discrimination:  individual  must  be  able  to  distinguish  legitimate  actions 

•  Proportionality:  actions  taken  and  response  must  be  balanced 

•  Necessity:  actions  must  be  just  in  cause 

Analogous  to  the  three  conditions  identified  above  relating  to  jus  in  bello ,  Christopher 
Cowley  argues  in  Moral  Responsibility  of  four  preconditions  necessary  for  assignment  of 
individual  responsibility:  capacity,  understanding,  control,  and  accurate  perception.57 

•  Capacity:  agent  must  possess  mental  capacity  to  discern  actions  and  events 

•  Understanding:  agent  must  comprehend  situation  connected  with  one’s  role 

•  Control:  agent  must  maintain  positive  awareness  of  situation,  be  cognizant 

•  Accurate  perceptions:  agent  must  possess  an  unaltered  consciousness  and  state  of 
mind. 

Accepting  that  human  cognitive  enhancement  does  somewhat  alter  one’s  perceptions  of 
reality,  thereby  potentially  affecting  judgement  of  relevant  facts,  arguably  there  is  some  risk 
human  cognitive  enhancement  may  negatively  impact  the  Just  War  concept  of  Jus  in  bello. 
Should  human  cognitive  enhancement  jeopardize  individual  responsibility,  corresponding 
accountability  would  suffer. 
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Human-Machine  Interaction  ...  “Locus  of  Control”  Concern 


Human-machine  interaction  further  raises  ethical  concerns  regarding  responsibility  and 
accountability.  Of  particular  concern  is  the  fundamental  imperative  regarding  “human  control” 
over  lethal  force.  The  pertinent  question  to  assess  is:  does  human-machine  teaming  (the  aspect 
of  interdependent  interaction  created  by  human-and-machine  teaming)  sufficiently  injure  human 
autonomy  to  void  assignment  of  responsibility.  Should  Airman-machine  teaming  and  flexible 
autonomous  weapon  system  designs  advance  to  where  it  become  difficult  to  discern  locus  of 
control  (is  control  of  action  due  to  human  thoughts  or  external  machine  response),  a  concern  for 
shared  responsibility  is  raised.  This  concern  is  compounded  should  the  human-machine  teaming 
scheme  include  human  cognitive  enhancement  clouding  responsibility  assignment.  Recognizing 
machines  cannot  be  held  liable,  the  concern  is  for  the  unwitting  creation  of  a  responsibility  gap 
with  no  party  capable  of  being  held  accountable  for  actions. 

LAWS  ...  Lethal  Autonomous  Weapon  Systems 

There  is  much  concern  regarding  the  use  of  autonomous  weapons  systems  in  war, 
especially  in  association  with  the  application  of  lethal  force.58  The  terms  “killer  drone”  and 
“killer  robot”  are  occasionally  used  to  refer  to  what  is  more  precisely  and  correctly  characterized 
as  a  Remotely  Piloted  Aircraft  weapon  system.  Disregarded  in  these  depictions  is  an 
acknowledgement  of  a  human  operator  providing  oversight  of  the  weapon  system,  and  human 
initiation  and  control  regarding  decisions  associated  with  the  employment  of  lethal  force.  Some 
have  argued  a  moratorium  or  ban  be  imposed  on  further  development  and  use  of  unmanned 
weapon  systems  until  a  full  review  of  ethical  issues  is  conclusively  settled.59 

Adding  to  the  above  concerns  regarding  unmanned  weapon  systems  are  questions 
relating  to  adoption  of  autonomy  and  artificial  intelligence  in  armed  weapon  systems.  The 
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subject  of  Lethal  Autonomous  Weapons  (LAWS)  was  recently  raised  at  the  United  Nations  (UN) 
Convention  on  Certain  Conventional  Weapons  in  May  2014. 60  One  of  the  concerns  expressed 
by  the  UN  surrounds  the  question:  what  constitutes  “meaningful  human  control”  of  an 
autonomous  weapon  systems.61  As  brought  forth  earlier  in  discussion  concerning  DoDD 
3000.09,  the  DoD  policy  on  human  operator  control  over  autonomous  weapon  systems  in 
respects  to  application  of  lethal  force  is  well  established.62  In  DoDD  3000.09  parlance, 
commanders  and  operators  have  a  duty  to  exercise  “appropriate  human  judgement.”  By 
extension  of  customary  international  law,  commanders  can  be  held  liable  for  improper  or  illegal 
actions  committed  by  operators  if  the  action  is  the  direct  result  of  commander  issued  direction.63 
At  this  juncture  it  is  relevant  to  review  the  philosophical  underpinnings  that  govern  conduct  of 
war  and  guide  discussions  regarding  characterization  of  human  control  as  DoDD  3000.09  states, 
to  exercise  “appropriate  human  judgement.” 

Explain  the  Meaning  —  Human  Control 

The  term  “appropriate”  human  control  as  applied  in  DoDD  3000.09  lacks  definition.64 
As  such,  there  is  vagueness  surrounding  what  constitutes  adequate  human  control  of  autonomous 
weapon  systems.  In  light  of  the  previous  discussion  surrounding  potential  ethical  concerns 
regarding  human  cognitive  enhancement,  this  uncertainty  fuels  a  debate  concerning  the  use  of 
autonomy,  and  in  particular  in  conjunction  with  lethal  weapon  systems.  For  some,  the  UN  effort 
to  qualify  human  control  is  in  effect  an  attempt  to  ban  the  use  of  unmanned  and/or  autonomous 
weapon  systems.  Understanding  some  may  seek  to  use  the  question  of  “appropriate”  or 
“meaningful”  human  control  to  restrict  the  use  of  autonomy  or  guide  policy  discussions 
surrounding  adoption  of  human  cognitive  enhancement,  the  DoD  and  AF  would  be  well  served 
by  exploring  the  issue  of  “human  control”  in  light  of  concepts  for  Airman-machine  teaming. 
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Recommendation 


As  the  AF  considers  pursuing  flexible  autonomy  (Airman-machine  teaming)  and 
adopting  human  cognitive  enhancement,  it  is  recommended  a  formal  study  effort  be 
commissioned  to  review  questions  and  address  concerns  relating  to  potential  ethical  implications. 

As  outlined  in  this  research  paper,  the  study  should  be  informed  by  scientific  researchers 
representing  all  relevant  sciences,  and  balanced  with  full  participation  and  views  heard  from  a 
wide  variety  of  concerned  stakeholders.  The  results  of  such  an  effort  could  serve  as  guiding 
future  research  and  supporting  science  and  technology  investment  decisions.  The  ensuing 
discourse  would  have  the  added  benefit  of  setting  “officially”  AF  policy  regarding  these  matters. 

The  study  effort  should  seek  to  provide  definitive  answers  to  the  following: 

1)  Are  there  ethical  implications  associated  with  human  cognitive  enhancement; 

2)  Does  human  cognitive  enhance  jeopardize  the  concept  of  human  control? 

Conclusion 

The  US  warfighter  will  face  a  highly  complex  security  environment  in  the  future.  The 
stakes  will  be  high.  A  multi-domain  environment  will  only  add  to  the  “cognitive  challenge” 
facing  weapon  system  operators.  An  Airman’s  future  wingman  could  very  well  be  a  machine. 

Science  and  technology  holds  promise  for  providing  capabilities  to  cope  with  the  mental 
demands  . . .  but  our  adversaries  will  not  rest.  Autonomy,  human-machine  teams,  and  cognitive 
enhancement  all  offer  potential.  Some  philosophers  argue  there  is  a  moral  obligation  to  pursue 
human  performance  enhancement,  yet  the  question  of  ethics  remains  ...  largely  unanswered.65 

As  science  and  technology  are  assessed,  scientists  owe  more  than  a  technology  roadmap. 
The  onus  for  charting  policy  is  a  shared  responsibility,  by  researcher  and  operator  equally.  For 
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policy  to  be  sound,  debate  must  be  fully  informed.  Stakeholders  must  guard  against  unconscious 
bias  and  rationalization.  The  stakes  are  just  too  high. 

Recognizing  much  uncertainty  exists  in  the  realm  of  neuroscience  and  brain-mind 
function,  scientists  must  continue  to  update  beliefs  with  relevant  scientific  knowledge.  As 
application  of  neurotechnology  in  weapons  system  designs  could  potentially  lead  to  significant 
consequences,  the  situation  requires  continual  examination,  discussion,  debate  and  dialectic 
discourse.  Policy  must  represent  varied  interests,  disciplines  and  views— from  all  stakeholders. 

As  the  DoD  and  AF  make  designs  on  future  warfighting  capabilities,  pursuit  of  combat 
advantage  must  remain  balanced  with  our  core  values  and  not  come  at  the  expense  of  ethics. 

For  the  US  warfighter  . . .  the  stakes  are  high! 
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